A graphical user interface (GUI) is one technology that allows a person to interact with an underlying application. However, it is sometimes beneficial to allow a process to interact with the GUI. The process may facilitate observing, manipulating, repurposing, and/or summarizing the application associated with the GUI. For example, a repurposing logic may be designed to modify a GUI of a website for mobile devices. In another example, a software testing logic may track and replay user inputs to a GUI. Logics that interact with GUIs sometimes associate a specific functionality to individual elements of a GUI. This may require that GUI components be classified into element types. However, it can sometimes be difficult for a logic to classify GUI components. Identifying how GUI components are related to other nearby GUI components adds another layer of complexity to this task.
Conventional GUI component classification techniques sometimes rely on information contained in a run time object hierarchy (e.g., a document object model (DOM)). However, in some cases an object hierarchy may not exist or may not contain useful information. For example, a FLASH® application in a webpage may contain multiple GUI components but the DOM for the website would classify the entirety of the FLASH® application as a single element. In another example, some Web 2.0 development toolkits may not adequately describe semantic information for some run-time objects. Thus, when hierarchy information is unavailable, conventional GUI classification tools may be unable to accurately label a GUI component.